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ABSTRACT 

The purposes of this paper are to (1) discuss general 
issues related to a newly proposed methodology for Item content and 
construct validation and (2) apply the new method in the context of 
developing a scale for measuring attitudes toward black people, 
uatent partition analysis was proposed and applied for the study of 
item content validity using partitioned data from 36 college 
presidents. Factor analysis was used to assess construct validity 
using item response data from 212 college students. A comparison 
between the item response factors and the LPA categories indicated 
substantial agreement. Limitations and virtues of the present 
methodology are discussed in relation to certain other methods for 
content and construct validation. (Author) 
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ABSTRACT 



The purposes of this paper are to (1) discuss general 
issues related to a newly proposed methodology for item content 
and construct validation and (2) apply the new method in the 
context of developing a scale for measuring attitudes toward 
black people. Latent partition analysis was proposed and 
applied for the study of item content validity using partitioned 
data from 36 college presidents. Factor analysis was used to 
assess construct validity using item response data from 212 
college students. A comparison between the item response fac- 
tors and the LPA categories indicated substantial agreement. 
Limitations and virtues of the present methodology are discussed 
in relation to certain other methods for content and construct 
validation. 




z 



Methodology for Instrument Validation: 
An Application to Attitude Measurement 



Robert K. Gable 
University of Connecticut 



Robert M. Pruzek 

State University of New York at Albany 
Introduction 

In the context of either attitude or achievement test 
development, two fundamentally different kinds of data are 
needed if derived scores are to be given valid interpretations: 
(1) judgments of subject-matter, or content, specialists 
regarding characteristics which items or tests are seen to 
measure, and (2) responses of individuals for whom the items 
or tests are considered to be appropriate. 

One purpose of this paper is to examine both judgmental 
and response data which were gathered in the process of devel- 
oping an instrument for measuring attitudes toward black people 
(Simons, 1971). Latent partition analysis (Wiley, 1967, Psycho- 
metrika) was used to study judgmental data gathered from 36 
presidents of colleges which are members of the United Negro 
College Fund. Factor analysis was used to examine item response 
data from 212 university students who were enrolled in black 
studies courses. 

A more egregious purpose is to discuss general strategy 
for developing tests. While the data for the present study 
are associated with attitude measurement, the discussion is 
relevant for the measurement of achievement as well as attitudes. 

Class if icatory Methods for Studying Content of Tests 

Unlike the traditional methods for studying response data, 
methods for describing rationally the content of attitude (or 
achievement) tests have not been thoroughly studied. The notion 
of examining item content using classificatory methods has 
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been inplicit in writings cf several measurement specialists 
(e.g. Lindquist et al, 1950), but not many empirical studies 
of such methods have been published* In the few reported 
studies of content validity, evidence for content interpreta- 
tions has typically been in the form of descriptive statistics 
indicating how adequately items fit a priori content categories, 

Robert Ebel published a paper in 1953 in which a method 
was described for classifying items into named categories where 
the categories distinguished different types of cognitive abil- 
ities. For this work, trained measurement personnel of the 
Examinations Services at the State University of Iowa regularly 
classified items from classroom tests into what Ebel termed 
"relevance categories," The practice was carried on simply 
for the purpose of describing test contents and no detailed 
analysis of these classifications has been reported by Ebel 
or his colleagues. 

A more systematic study of this type was carried out by 
McGuire (1963) in a study of certifying examinations in medical 
education. McGuire had several subject-matter specialists 
categorize items from medical certifying examinations into 
categories roughly equivalent to the levels of Bloom’s (1956) 
taxonomy of the cognitive domain. Results were presented 
simply in terms of descriptive statistics of agreement among 
the various categorizations. Certain agreements were found 
among the categorizations but some substantial discrepancies 
were also noted. McGuire also employed a factor analysis to 
analyze correlations between the eight subtests defined (appro- 
ximately) by he?' categorizations of items, but most correlations 
were low and derived factors were not interpreted, 

A more sophisticated classificatory study of achievement 
test items was that of Stoker anti Kropp (1964). These authors 
also used a categorical system derived from the Bloom taxonomy. 
Items were devised to correspond to the major levels of the 
taxonomy and then persons familiar with the various levels were 
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asked t r- sort items independently intc the respective named 
categories . Explicit attention was given to the complexity 
continuum on which Bloom* s categories are ordered. As in 
McGuire *s study, results of the independent categorizations 
were reported simply in terms of descriptive statistics of 
agreement, and again, numerous discrepancies were found between 
resultant categorizations. Subsequently, relationships between 
results of the categorizations and results of response data 
analysis were discussed briefly. 

Although the classif icatory procedures discussed above 
surely constitute what may be termed ad hoc methods , the pro- 
blems raised by the studies are significant. We shall consi- 
der such problems in the following section. 

An Analytic Method for Item Content Validation 

Measurement specialists typically do not regard content 
validity as primarily a quantitative characteristic of tests. 
Rather, this type of validity seems more properly viewed as 
qualitative. It is for this reason that methodology for item 
content validation may properly involve item classifications 
or categorizations. In this part of the paper we shall be 
concerned with the description of a method called latent 
partition analysis (LPA) for specifying content validity of 
attitudinal measures. Although we focus on LPA in this paper, 
it should be emphasized that our proposal rests more on the 
logic of a strategy of gathering data, and making decisions 
directly with respect to this data, then with any particular 
feature of LPA itself. The objectives and limitations of 
the proposed approach will be briefly considered in relation 
to features of typical methods of instrument development. 

Latent partition analysis was formulated in connection 
with research 1 in which a method was needed to study relation- 
ships between different sorters* categorizations of the same 
set of items. For that research the items were statements 

^Project No. 5.1015.2.12.1 (1967), supported by the U. S. Office 
of Education; Principal Investigator, Donald M. Miller, Univer- 
sity of Wisconsin. 
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pertaining to teacher facilitation of learning in the elemen- 
tary classroom which had been obtained through interviews 
with elementary teachers. Following the interviews, each of 
a number of teachers were given the same set of statements 
and were asked to sort these into what they considered homo- 
geneous categories. Latent partition analysis (Wiley, 1967) 
was formulated and applied to study relationships among the 
resulting sets of categories (partitions). 

Stated in Guttman's (1950) terms, the critical problems 
of building internally consistent tests are to define properly 
the intital universe of content and to select properly items 
for this universe. Guttman suggests that this should be a 
logical (deductive) process, but there are some formidable 
problems with the approach he describes. It is generally 
very difficult to know just what constitutes a particular 
universe, when two or more universes may bo merged and when 
a universe should be broken down into subuniverces • In addi- 
tion, once a universe has been defined, it may be difficult 
to generate items which in fact measure that which has been 
specified. 

In order to circumvent the above problems, or at least 
to handle them more objectively, it is proposed that an 
inductive approach be used in generating internally consistent 
tests (or sub^ests) • The specific suggestion is to begin 
from a pool of attitude (or achievement) items, to have each 
of several qualified persons sort the items into what they 
deem as homogeneous categories and to use an analytical method 
such as latent partition analysis (LPA) or hierarchical cluster 
analysis, to derive a single set of homogeneous item clusters. 

We propose that under certain conditions at least, content 
labels might be inductively derived from such a process; the 
items or labels can be modified at subsequent stages of research 
in order to refine an instrument. 
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Suppose that several subject-matter specialists were asked 
to sort independently a set of attitude (or achievement) items 
into homogeneous categories. Let us assume that no prior 
(named) categories have been specified. One would expect 
different persons to construct different categories; neverthe- 
less, if each sorter had been given the same instructions for 
creating his categories, one would expect to find relation- 
ships among resulting categorizations. It is proposed that 
LPA be used to examine such a set of categorizations with 
the objective of discovering underlying agreements about 
contents which might not otherwise be evident. If each sorter 
were asked to generate categories on the basis of the content 
characteristics of the items, it seems reasonable tc expect 
derived categories to reflect content characteristics of the 
items. That is, items within derived categories should be 
relatively more homogeneous in content than items between 
categories. Tucker (1962) noted that the critical problem 
in establishing content validity for achievement (or attitu- 
dinal) test items is specification of the universe of content 
for which items may be regarded as valid measures. Different 
qualified persons may be found not to agree about the parti- 
cular universe (or universes) for an item. As described above, 
LPa or a similar method such as Johnson* s (1967) hierarchical 
cluster analysis, may be used operationally to define content 
universes for items 

In his chapter entitled Test Validation, Cronbach (1970, 

P. 446) essentially notes that the principal question of con- 
tent validity is: ”Do the observations [items] truly sample 

the universe of tasks [attitudes] the developer intended to 
measure?” The LPA method approaches this question by asking: 
”What are the constituent subuniverses for a universe which 
may be inductively derived from constructed items?” By 
examining derived LPA categories, and their mutual interre- 
lationships, subuniverses are identified; then the subuniverses 
are studied in relation to the investigator’s initial concep- 
tion of the universe; items are added to or taken away from 
the original item pool on the basis of the above analysis. 
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The most important feature of the proposed approach is 
its relatively heavy reliance on objective methods for the 
finding and naming of content categories. Unlike most methods 
for classification, such as those reviewed, the present stra- 
tegy involves an objective solution for item categories, even 
when the investigator has no named categories at the onset. 

While we agree with Rozeboom (1966) that our whole notion of 
content validity may require revision, and that the term 
"content” simply isn’t a theoretical term, our approach seems 
at least to provide a better framework than most methods for 
ultimately interpreting item response data; this is our primary 
goal • 

Methods for Studying Constructs of Tests 

While assessments of content validity are based more or 
less exclusively on judgmental data, construct validity is 
based on examinations of judgmental data and of response data. 
Cronbach (1970) provides an enlightening discussion of con- 
struct validation as a continuing process of refining and 
elaborating on test (score) interpretations. In general, 
the process can be viewed in the context of three procedures : 
labeling hypothetical constructs, formulating testable 
hypotheses on The basis of some underlying theoretical network, 
and gathering evidence to test the hypotheses • 

The initial labeling of The constructs consists of employing 
a method such as LPA to generate detailed descriptions of 
item content characteristics in the manner previously described. 
Following this, both confirmatory and exploratory approaches 
to factor analysis can be employed to answer the questions 
"Are factors which have been hypothesized in advance using 
judgmental data sufficient to account for the stable interitem 
covariation of the item response data?" Note the emphasis on 
relationships between content and construct validition. As 
a rule, when item responses have been influenced substancially 
by characteristics of the items which were not weighted heavily 
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in the judgmental data study one may expect discrepancies 
between LPA categories and response data factors. It is the 
detailed analysis of the nature and extent of such discrepancies 
which seems ultimately to be likely to add to knowledge ol the 
constructs under study. While item response data may in general 
correspond to mere conceptualizing habits of a culture or to 
accidents of simultaneous learning of different constructs , 
and judgmental data categories may in turn be influenced by 
capriciousness of judges* perceptions , we believe that there 
is capital in making discrepancies explicit between these 
two fundamentally different kinds of information. Careful 
sampling of both judges and respondants will of course be 
most important in making studies such as these productive. 

At this point we illustrate the suggested approach to 
instrument development in the context of a study by Simons 
(1971) which deals with the development of an instrument to 
measure attitudes toward black people. 

LPA for Contenr Specification: 

An Empirical Study ” 

In this section an empirical study by Simons (1971) which 
employed LPA to examine the content validity of an attitude 
instrument will be described. 

Procedures 

The categorical data examined in this section resulted 
from having 36 content specialists independently sort 51 items 
into categories. The items sorted reflected an item pool which 
had been generated on the basis of information obtained from 
literature reviews including journals , newspapers , magazines, 
radio, television, and lectures. Preliminary item screenings 
and data analyses by graduate students suggested that 51 items 

were worthy of further study. 

The 36 sorters were presidents of colleges and univer- 
sities located in Southern and border state that were members 
of the United Negro College Fund. The sorters were selected 
because of their association with faculty, alumni, and student 
groups which were predominantly black. 
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Items were put on individual slips of paper (without 
? tem numbers) and mailed along with instructions to sort the 
items into from 6 to 15 mutually exclusive and jointly 
exhaustive categories on the basis cf similar item content. 

Judges’ categories, called manifest categories, were 
used as the starting point for latent partition analysis 
(LPA). The LPA program computes a joint proportion matrix 
(of order 51 x 51) where each entry indexes the proportion 
of (36) sorters who placed a given pair of items in the same 
manifest category. From this matrix, a latent category matrix 
is derived; entries in this matrix index for each item the 
derived (latent) category to which the item belongs. 

Results of the LPA Study 

Table 1 presents the latent category matrix; rows repre- 
sent items and columns, categories. Inspection of the entries 
in Table 1 indicates that 12 latent categories were derived. 



Insert Table 1 About Here 

and that, for the most part, items are of unit complexity. 
Thus, we regard the LPA model as reasonable for these data. 

The following is a category-by- category description of each 
category in terms of what we call item content. Although we 
have taken the liberty of briefly naming each category, the 
reader should recognize that some liberties h.t r e been taken 
and that subsequent tables (especially 2 and Appendix A) 
ought also to be examined. 

Category 1 was called Racism , since the item content 
states that black people have low moral standards and are 
innately inferior. Category 2 was named Black Studies Courses 
all items in this cluster described various aspects of the 
objectives and characteristics of such courses. The third 
category, Inner City Teachers , was very clearly defined by 
items describing characteristics of inner city teachers. 



10 



0 



Page 2 



Category 4 was called Public Opinion as all three items 
referred to the views of the university community toward blacks. 
The next category * was labeled Community Control ; it was 
neatly defined by items suggesting community control of 
education, police and business. Also clearly defined was 
Category 6, Philosophy . Items in this category dealt with 
the credibility of such organizations as the NAACP , SCLC and 
CORE. 

Items reflecting characteristics of black students 
defined Category 7, Student Image . Category 8, was termed 
Cultural Differences since the item content suggested the 
existence of black art, theater and music* Although Category 
9 was called Race Interaction , the three items defining the 
category were deleted from the study. Several judges commented 
that the items were inaccurate and potentially offensive. 

Category 10, Personal Judgment , was defined by three items, 
one of which was offensive and thus deleted. The remaining 
two items reflected awareness or judgments of other people 
due to their skin color. Category 11 was called Cultural 
Background but was not clearly defined. Two of the three 
items defining the category were deemed offensive; the third 
item reflected possible teacher prejudice where a black 
student’s classroom behavior tends to be rated low even 
though his academic achievement is satisfactory. Finally, 
Category 12 was called Bigotry since the items reflected the 
view that black students are inarticulate and apathetic. 

The latent category matrix itself suggested that several 
clearly defined subuniverses can be identified for the judg- 
mental data. Table 2 contains the matrix of indices of asso- 
ciation between the pairs of latent categories. Inspection 

Insert Table 2 About Here 



of the entries suggested that some content categories reflected 
similar subuniverses of item content. Given this situation 
it was reasonable to combine some of the item categories to 
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produce a more parsimonious system of content clusters. Thus 
the following three sets of categories were examined for pos- 
sible combination; Racism (1, Racism and 12, Bigotry), Fublic 
Opinion (4, Public Opinion and 10, Personal Judgment), and 
Black Studies (2, Black Studies and 8, Cultural Differences). 
Examination of the three categories indicated that the first 
two wore quite consistent, but the tendency of the judges to 
combine the Cultural Differences items with Black Studies 
merely suggested the judges association of black art, theater 
and music with possible course topics in Black Studios. Since 
the instrument being developed was designed to reflect atti- 
tudes towards black art, theater and music, the Black Studies 
and Cultural Difference categories were not combined. 

Thus our LPA results were taken to reflect eight cate- 
gories of item content for the 51 items. Forty-one of the 
51 items contributed to the naming of the categories in the 
latent category matrix. A total of six of these items were 
deemed inaccurate and possibly offensive by the judges and 
the researcher. To increase the internal consistency relia- 
bilities of the scales on the instrument under development, 

15 new items were written for selected categories to generate * 
a 50 item instrument. Table 3 contains a summary of the 
original and final category labels and illustrates where 

Insert Table 3 About Here 

items were deleted or added on the basis of the LPA study. 

The original derived categories are designated LC's 1-12; 

the final judgmental categories are referred to as FJC I to FJC VIII. 

This notation will be used to distinguish between original 

and final categories. 

In summary to this point, the LPA procedure provided a 
rational-inductive identification of the constituent sub- 
universes of item content. Examination of the derived cate- 
gories, *:he relationships between categories, and the intended 
conception of the universe of content suggested certain item 
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deletions and additions with respect to the original item 
pool. A comparison between the eight LC T s which were empi- 
rically generated on the basis of judgmental data, and actual 
item response factors will add further information regarding 
interpretation of the attitude constructs. Such a comparison 
will be presented in the next section. 

A Factor Analysis of Response Data 

After analysis of judgmental data, and suitable revisions 
of attitude items, an instrument comprised of 50 attitude items 
using a five point Likert format was administered to 212 uni- 
versity students who were enrolled in a black studies programs , 
or other courses designed to relate to this program. Some 
of the 50 items had been written to have negative stems while 
others were positive; all negatively stated items were ^arbi- 
trarily) reflected in the scoring. 

Using these data a 50 x 50 matrix of inter correlations 
was computed from which we generated principal components 
and followed with a normal varimax transformation. These 
results are presented in the following section with a des- 
cription of relationships between the response data factors 
and the LC's from the LPA study. While we do not in this 
context present detailed analyses of the attitude constructs , 
an analysis is available in Simons (1971) and Simons ana 
Gable (1S72). (We also analyzed the CO x 50 matrix using a 
modern version of scale-free image analysis but the factor 
interpretations were virtually identical tr those of the 
derived components solution so wc have chosen to include only 
the components results here.) 

Results of Factor Analysis Study 

Sixteen factors (components), accounting for 66% of the 
total variance were derived; nine of these were defined by 
at least two items with substantial loadings and, we there- 
fore present a nine factor solution. The factor loading 
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matrix can be found in Appendix A. Table 4 contains the 
factor names, LPA original category (LC's 1-12) codes, LPA 
item numbers (for comparison with Table 1), the 50-item 
version item numbers, item summaries and factor loadings. 
Inspection of the entries in Table 4 indicates tne extent 
that the item response factors reflect LPA item content 
categories . 

Insert Tabic 4 About Here 

Factor I was called Racism since it was defined by six 
of the eight LPA racism items from LC. Note that these six 
Racism items consisted of two items from the original Racism 
category, two from the original Bigotry category and two 
new Ricism items. People tending to agree w5.th the content 
of the items defining this factor might be considered to 
have racist attitudes. 

Factor II was labeled Inner City Teachers; it was 
clearly defined by the seven Inner City Teacher items from 
FJC III. Recall that Table 3 indicated that FJC III con- 
sisted of five original Inner City Teacher items, one new 
item and one item from the original FJC II, Cultural Back- 
ground. All seven of these items, grouped on the basis of 
judgmental data in the LPA study, were sufficiently inter- 
related for the student response data to generate Factor II. 
Agreement with thu content of these items by a respondant 
suggests understanding and support of inner city teachers • 

Factor III, Cultural Differences, was defined by all 
five of the items judged to form FJC VIII, Cultural Differ- 
ences. Agreement with the item content acknowledges aware- 
ness of such cultural aspects of the black experience as 

dance , music, theater and art. 

Factor IV was called Public Opinion since it i.as defined 
by five of the ten Public Opinion items from FJC IV (see 
Tables 3, 4). Of the remaining five items, one item formed 
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a single item factor and was therefore deleted > two items 
each clustered to form Factors VI ; Personal Judgment and 
Factor VIII, Public Opinion: Personal Involvement. It will 

facilitate the discussion to describe Factors VI and VIII 
before commenting on Factor IV . 

Factor VI was labeled Personal Judgment since the two 
items defining this scale reflect an individual's general 
(positive) attitude towards blc.ck people. 

Factor VIII was named Public Opinion: Personal Invol- 

vement because the two items defining the factor reflect a 
specific-personal aspect of public opinion. 

The items defining Factors IV. VI and VIII were judged 
in the LPA study (see Table 3) to measure Public Opinion 
(LC 4) and Personal Judgment (LC 10). The items defining 
these two LC's were grouped by us to form the FJC IV, 

Public Opinion. The appropriateness of this item grouping 
can b-i examined in light of the interrelationships uncovered 
among items using factor analysis. Consideration of the dis- 
crepancies between the judgmental and response data adds 
information to the interpretability of the intended LC. 

Public Opinion For example, the judges tended to categorize 
the items reflecting rather general opinions of whites toward 
black people, a specific individual's opinion toward a black 
person as well as opinions possibly reflecting more persona) 
involvement, into a general Public Opinion category (FJC IV). 
But the response data dimensions suggest that interpretation 
of these public opinion items would be more meaningful in 
the context of the three aspects of public opinion reflected 
±.n Factors IV, VI and VIII. 

Factor V, Philosophy, was clearly defined by five of 
the six items from FJC VI, Philosophy. It is interesting 
to note that agreement with the items of this dimension 
indicates a respondant's support of 3CLC, CORE and NAACP 
but not the Black Panthers. The final Philosophy item dealing 
with the credibility of the Black Muslims did not contribute 
to naming this factor. 
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Factor VII, Community Control, was defined by the four 
items from FJC V. Agreement with the item content reflects 
support of black community control of business, police, 
education and private industry. 

Finally, Factor IX has been labeled Student image 
because it was defined by three of the five items from FJC VII , 
Student Image which mainly reflected views of teachers regarding 
the classroom behavior of black students. The remaining two 
Student Image items, reflecting personal qualities of black 
students, loaded across separate factors which were not 
selected for discussion. 

It should be noted that these data do not warrant res- 
ponse data clustering of the five items from FJC II, Black 
Studies Courses. 

Comparison of LPA and Factor Analytic Data 

In this section additional comments are made concerning 
relationships between the results of judgmental data and 
response data analysis. 

With the exception of the Black Studies Courses items, 
substantial agreement was found between the response data 
factors and the judgmentally derived categories. Examina- 
tion of discrepancies between the categories and factors 
(e.r. FJC IV Public Opinion and Factors IV, Public Opinion 
VI, Personal Judgment and VIII, Public Opinion: Personal 

Involvement) contributed to a better understanding of the 
attitudinal constructs under investigation. Consideration 
of the am ount of agreement between the categories and fac- 
tors for these data in light of the internal consistency 
reliabilities of the factors suggests that reliabilities 
tend, as expected, to be higher for those scales with more 
items and for the subscales which corresponded jointly in 
judgmental and response data clusters . 
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Finally, it should be noted that additional analyses of 
response data illustrated in this study were carried out, 
but not reported in this paper. A confirmatory method of 
factor analysis, Guttman ? s (1952) rank reduction procedure, 
was employed by Simons (1971) to empirically examine the 
response data after an a priori hypothesizing of the specific 
item clusters on the basis of the LPA judgmental data. 

Results of the analysis were generally supportive of the 
interpretations found in exploratory principal components 
analysis. Also results of an oblique rotation of the prin- 
cipal component loading matrix and an image analysis followed 
by both orthogonal and oblique rotations (Hofmann, 1970) 
were found to agree with the factorial interpretations which 
have already been reported. 

Discussion and Conclusions 

In this paper it has been proposed that objective methods 
(using LPA) may be employed to identify certain subuniverses 
within a universe of items. Given an appropriate selection 
of judges (sorters), items, and sorting directions, one might 
expect individual derived item categories to be predictive 
of ultimate response data categories. In turn, this strategy 
might be sequentially applied as item pools are modified, 
and as new response data is collected. This sequential process, 
may be regarded as a means to assist in construct validation 
since naming of one's ultimate item categories should be 
facilitated by an examination of relationships between judg- 
mentally derived and response data derived clusters. As 
Cronbach (1971) notes, both item and test validation are 
fundamentally a process of interpretating test scores, and 
findings those interpretations for which an item (or test) 
is valid. There are strong parallels between attitude and 
achievement test development so distinctions between these 
types of tests seems not to be crucial. It seems reasonable 
further to examine the present methodology m relation to 
facet systems for item content specification although such 
a task will not be attempted here. 
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It is not suggested that the LPA method is necessarily 
sufficient for judgmental data analyses , but it does appear 
to fill a certain chasm in the spectrum of typical test 
developmental methods. If additional empirical studies of 
items in a content domain do show that there is a general 
correspondence between derived LPA categories and results 
of response data analyses, numerous furtner uses of the 
proposed strategy become obvious. Construction of ’’strictly 
parallel” forms and score equating across forms are cases 
in point. It must be reiterated, however, that such uses 
will require empirical study of relationships between results 
of categorical data studies and response data studies for 
the same items. Also, it may be quite important to select 
one’s method for analyzing response data characteristics. 

Some comments on sampling problems related to LPA 
studies are also necessary. Three fundamentally different 
typos of sampling are involved in any judgmental study; 
one must select items, sorters and sorting instructions. 

An investigator will, of course, be prudent to restrict his 
attention to a pool of high quality items (those for which 
agreement exists that some relevant characteristic may be 
measured). Results of a judgmental study may be meaningful 
explicitly only within the context of the particular items 
chosen. Items should not be overly complex in form, nor 
should they usually contain more than one principal idea. 

To the extent that items are highly complex, the likeli- 
hood for interpretable results is apt to be substantially 
diminished. 

The initial selection of persons to sort the items can 
also be critical. Certainly some persons might produce 
categorizations greatly different from those of other per- 
sons. Sorters should probably be selected to have experi- 
ence or competence in the subject-matter area if categories 
are to be most interesting. For some ideas on how one might 
study relationships among different persons’ categorizations 
of the same items see Tucker and Messick (1963) and Pruzek, 
Stegman and Pfeiffer (1972). 
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The last problem to be considered is that of selecting 
instructions for sorting. Instructions in the present em- 
pirical study were designed with the intent of deriving 
categories which reflected content characteristics of the 
items. It is possible that different instructions, even if 
generally based on the principle of content differentiation, 
might have resulted in different derived categories. While 
such a result appears unlikely to these investigators; it 
remains a possibility. More generally, we can conceive of 
entirely different bases for category generation, e.g., 
item content complexity, which could be fruitful. All of 
these latter questions, as well as those relating to the 
sampling of persons and sorters, must be viewed as questions 
for further study. 
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Appendix A 



Component Loading M-trix Using A 
Varimax Rotation For Simons Data* 
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*Xii entries have been multiplied by 100. Note that factors 1-7 and 9-lb 
were the nine factors chosen for interpretation. 
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Derived Approximation to Latent 
Category Matrix for 51 Attitude Items 
with 12 Categories* 
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•Kows were automatically reordered to facilitate interpretations; 
All entries have been multiplied by 100. * 
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TABLE 2 



Indices of Association Between 
Latent Categories for Simons LPA Results* 
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*Entries in this matrix will lie between zero and unity 
when the model fits the data. If the matrix is essen- 
tially diagonal, most manifest categories can be said 
to result from differential splitting of the same latent 
categories. Diagonal entries estimate the probability 
that any two items in that category will, in a new 
partition, be sorted into the same manifest category 
off-diagonal entries estimate the probability that two 
items from two different latent categories will be 
placed in the same manifest category. 
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TABU I 



('.ii'tof , «'l igm.i l i n 1 tv"> It'" 1 ''tens 

and l‘v'r;>\'lu‘i»t Loading* I v: iVt \\'n.j»oac:it Sol .it !**•»• 



factor 



I 

R.ieisn 



1 1 

! mu’ r 
k i t v 

Teacher* 



til 

Cultural 

Differences 



IV 

Public 

Opinio; 



V 

Phi losophy 



VI 

Personal 

Judgment 



.'rigiu.il 
IPA .ode 


1 ten 


Sten Sugary 

. .. - . 


l oa Jiii 


K 


13 


:o 


nerber> v>f K.uk race h.ivc lew moral 
standards 


• * 


P 


;o 


3* 


black >tuden:$ inarticulate 6 now verb.* 


* M 


B 


s «► 


41 


black students apathetic* Jell - selJor: 
am »atco 


eJ 


R 


14 


:s 


majority hi **ks tend to be lutv-r.ither 
not * .* r k 


4 a 


R 


new 


54 


ethn;c groups Jitter \-\ ;nnati- ab;iitv 


4 ;• 


R 


new 


4 0 


blacks : ot l . k ii.Jn iU.;al initiative 


4 0 


r'i 


row 


5* 


philosophy o: b lack Panther? no t creai. 


U 5* 


ir 


54 


11 


no concern for welfare of blacks tiie^* 
teach 


s: 


IT 


35 


* 


concerned w i t‘.i salaries :*.ot needs ot 
iniiv.Jutl 


Sv' 


ir 


5 S 


:i 


inner c;ty >*chocis-d..r pir.g gi‘o..:iJ> tor 
inaJccviate teachers 


ee 


IT 


row 


:9 


Jo not .rival' e the* selves with 
cor.mr: i ty agencies 


to 


C3 


5o 


45 


rate be avior of Mack lew e\e~ it 
achievement is s it istactcrv 


tO 


IT 


4S 


45 


experience difficulty in ur.JerstanJinj 
blacks 


5 2 


IT 


53 


55 


teaching position* in inner city not 
at tr 30 t able 5 ar.bitious teachers 


4 5 


CD 


r.ew 


50 


there is black dance 


85 


CD 


:o 




there i? Mack -u*ic 


81 


CP 


19 


IS 


there is black theatre 


80 

78 


CD 


13 


s 


there is black art 


CD 


new 


43 


there is black folklore 


56 


PO 


49 


7 


whites at university level sympathetic 
to the protlers of blacks 


SI 


PO 


50 


1? 


whites at university level want equalit 
for blacks 


81 


PO 


new 


42 


if blacks given equal opportunity, 
probably make success of live? 
university attempts neet biack needs 


70 


PO 


51 


25 


53 


P0 


new 


33 


whites in university believe 1 : blacks 
given good job* •'.akc a success of li\os 


Ph 


44 


15 


philosophy of SCLC is credible 


74 


Ph 




31 


philosophy of Operation 3read iasket 
credit ie 


75 


Ph 


43 


4 


philosophy of NAACF is credible 


42 
* «* 


Ph 


45 


23 


philosophy of CORE not credible 




Ph 


new 


3 : 


philosophy of Black Panther not credibi 


o - 36 


PJ 


41 


19 


when interact with blacks, r.ot aware of 
skin color 


73 


PJ 


40 


9 


if black were president of U.S., not 
forget skir. color 


38 



VII 

Conmuni ty 
Control 



CC 31 
CC 30 
CC 29 
CC new 



15 community control of education desirable 

black community 

16 community control of business desirable 

black "conmur.it/ , . 

6 community control of police desirable 

black community 

49 community control cf private industry 

not essential in black community 



73 

70 

67 

46 



VIII 

Public 

Opinion: 

personal 

Involvement 



P0 new 46 if newborn black baby adopted into 
black middle class family, learn 
middle class values 

P0 new 27 if newborn black baby adopted into 
white niudle class family, learn 
middle class values 



83 

77 



IX 

Student 

Itage 



SI 2S 30 

SI 22 3 



SI new 



44 



way to maintain discipline is to be 
strict ( direct 

black students discipline problem; 

toughness • conic activities 
black*, generally civilised as whites 



70 

66 

S6 



th-jt Tor eiTTi item the 
arc listed for comparison * 
p.-icisB. was originally item 
study. 



origin*? I.I-A category code and item number 
ith Table 1. for e/ar.f»Jc, item 20 in factor I 
13 in the category labeled Racism in the I.PA 
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TABLE 5 



Factor Name, Number of Items Corresponding to LPA 
Suggested Categories, Number of Items Defining Each Factor, 
and Estimated Alpha Internal Consistency Reliabilities 



Estimated 





Factor 


Number of Items 
Corresponding to 
LPA Suggested 
Categories 


Number 
of Items 
Defining Each 
Factor 


Alpha Scale 
Internal 
Consistency 
Reliability 


1. 


Racism 


6 


7 


.76 


2. 


Inner City Teachers 


7 


7 


.32 


3. 


Culture Differences 


5 


5 


CO 

• 


4 . 


Public Opinion 


5 


5 


.77 


5. 


Philosophy 


5 


5 


.64 


6 • 


Personal Judgment 


0 


2 


.23 


7. 


Community Control 


4 


4 


.71 


3. 


Personal Opinion: 

Personal Involvement 0 


2 


.47 


9. 


Student Image 


3 


3 


.59 
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